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Measuring Student Engagement, Knowledge, and Perceptions of 
Climate Change in an Introductory Environmental Geology Course 
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ABSTRACT 

This research examines a semester-long introductory environmental geology course, which emphasized climate science using 
an Earth systems approach and employed a multipronged teaching strategy comprising lecture, movie viewing, class 
dialogues, and journaling. Evidence of student engagement during various pedagogical approaches (e.g., movie viewing, 
lecture, and class dialogues) was measured using skin sensors on a subset of student participants in order to gauge student 
engagement. Results indicated that students are more engaged during movie viewing and dialogue than during lecture, with 
measurable increases when climate solutions and local impacts were covered. Qualitative data including journal entries and 
transcripts from student dialogues were analyzed to determine patterns about student climate change perspectives and 
discourse for which students' self-reflections supported quantitative measures. Measurement of student pre- and post-climate 
change knowledge, confidence, and perceptions revealed that the combined pedagogical approach supported student learning 
gains about climate science. © 2014 National Association of Geoscience Teachers. [DOI: 10.5408/13-111.1] 
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INTRODUCTION 

Research indicates that increases in globally averaged 
temperatures of just a few degrees in this century will likely 
cause an increase in the occurrence of drought, floods, and 
extreme weather and accelerate sea-level rise into the future 
(IPCC, 2007a, 2007b, 2007c, 2013). Therefore, addressing 
climate literacy in higher education, especially within 
courses that house geoscience majors, is important so that 
future scientists can form evidence-based decisions, develop 
strategies to address climate change, and communicate with 
the public effectively about the issue. However, understand¬ 
ing climate change requires grasping systems thinking and 
understanding of change over multiple spatial and temporal 
scales (Rebich and Gautier, 2005; Sell et al., 2006; McNeal et 
ak, 2008). Students often have difficulties with understand¬ 
ing radiative processes, water vapor, feedbacks, and the 
timescales over which climate change can occur (Gautier and 
Rebich, 2005). Some students believe the greenhouse effect 
is due to the ozone hole or that greenhouse gases are "bad" 
rather than necessary for moderating Earth's climate (Boyes 
and Stanisstreet, 1994; Gautier and Rebich, 2005). Further¬ 
more, some students may use localized weather events to 
draw conclusions about climate change, which occurs 
regionally and over much longer periods (Choi et al., 
2010). In order to assist learners in reasoning about climate 
change, new ideas must be integrated with preexisting 
conceptual models (Vosniadou and Brewer, 1992; Bransford 
et al., 1999; Chi, 2005). 
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People often default to reliance on their fundamental 
values and worldviews when learning (McCright and 
Dunlap, 2011; Leiserowitz et al., 2012). Incoming informa¬ 
tion that addresses climate change may be rejected upon 
quick judgment if it challenges deeply held beliefs of an 
individual or those of the group with which that person most 
identifies (Kahan et al., 2007, 2012; Kahan and Braman, 
2008; CRED, 2009; Rutherford and Weber, 2011; Lombardi 
and Sinatra, 2013), making teaching about climate change 
challenging. 

Emotions can be tied to engagement, which can strongly 
influence learning about a topic (Irwin and Wynne, 1996; 
Ockwell et al., 2009; Weber, 2010; Lombardi and Sinatra, 
2013). Pintrich et al. (1993) introduced the concept of hot 
versus cold conceptual change—cold conceptual changes 
occur only by engaging the cognitive domain, whereas hot 
conceptual change involves engaging both the cognitive and 
the affective domains to bring about lasting changes in 
learners' preconceptions. Further research has shown that 
academic emotions play an important role in student 
learning (Pekrun et al., 2002), and emotional states have 
been found to be interlinked with cognitive processing 
(Linnenbrink and Pintrich, 2004; Goetz et al., 2010). 

Direct measurement of student engagement in class¬ 
room settings and associated affective measures can be 
challenging. Post hoc questionnaires have been widely used 
for this purpose. However, such techniques can be limited in 
scope because of the qualitative nature of the response and 
because a delayed self-response to an emotional state could 
bias such data. The advent of affective computing has the 
potential to circumvent this problem by providing continu¬ 
ous and potentially unbiased physiological data that can 
open a window into the engagement of research subjects to 
external stimuli. 

Recently, a new methodology for measuring engage¬ 
ment in education research, called galvanic skin response 
(GSR), has been introduced in science education (Poh et al., 
2010). Skin conductance or GSR is the electrical response of 
skin at low voltage and is usually measured with a pair of 
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FIGURE 1: Smooth SCL (tonic phase) is interrupted by a 
rapid rise in skin conductance (SCR, or phasic phase) 
due to a stimulus. Subsequently, the skin conductance 
attains a new SCL, which is usually higher than the 
prestimulus level. 

electrodes touching the skin of the palm or the wrist. Skin 
conductance is a function of the moisture level of skin, which 
in turn is controlled by sweat gland activity. As sweat glands 
are a part of the sympathetic nervous system, which also 
governs our emotional states, it has been postulated that an 
increase in sweat glands activity signifies both psychological 
and physiological arousal (Malmivuo and Plonsey, 1995). 

GSR is a composite of a slowly varying or tonic 
component, called the skin conductance level (SCL), and 
separated by peaks, the phasic skin conductance response 
(SCR). Typically, SCLs are interrupted by SCRs due to an 
event or stimulus, and the skin conductance increases 
sharply and then decays slowly to a new SCL (Fig. 1). SCLs 
can be used to infer average level of psychological arousal or 
engagement over a sustained period (hours) or to compare 
engagement between types of stimuli (e.g., watching movies 
versus listening to lectures). However, they are not suitable 
to indicate responses to short-lived stimulus and bursts of 
activity (e.g., a startling sound or a particularly revolting 
imagery). For such cases, phasic SCRs are more appropriate. 
The SCRs are considered to be the responses to bursts of 
activity in the sudomotor fibers (part of the sweat glands), 
and the amplitudes of the SCRs are correlated with the 
number of sweat glands involved in such bursts and the 
strength of the sudomotor activity in those glands (Benedek 
and Kaernbach, 2010). In other words, an affective state 
change, for example, a reaction to a particularly pleasing or 
offensive sensory stimulus, can be manifested as a SCR 
(from bursts in sudomotor activity, which controls sweat 
generation). The SCRs usually show a lag time after the 
stimulus (1-5 s) and have minimum amplitude of 0.01-0.05 
/(s (Dawson et al., 2007). 

Some issues exist with collecting GSR data, such as the 
skin-sensing unit worn by the participant potentially 
introducing difficulty in the data collection process and the 
often study-specific identification of a meaningful SCR to a 
stimulus (Dawson et al., 2007). However, these obstacles can 
commonly be controlled and minimized, especially when the 
participant's movements are kept to a minimum, reducing 
the sweat that can accrue in the sweat gland ducts of the 
skin, which are the primary sites for GSR data collection 
(Potter and Bolls, 2011). 

Skin conductance studies have become commonplace in 
psychological research due to their reliability in detecting 
GSR changes that are commonly associated with emotional 
processing of stimuli such as pictures, music, video games, 


and film (Potter and Bolls, 2011). Skin conductance methods 
are often utilized in place of other psychophysiological 
measures due to the ease of measurement and analysis of 
GSR data (Andreassi, 2007). As such, skin sensors can form 
an important part of a larger set of affective computing gears 
that seeks to monitor biophysical responses such as heart 
rate, muscle activity, skin temperature, blood volume pulse, 
electrocardiogram, and respiration (Haag et al., 2004). For 
example, affective e-learning, a collaborative research project 
between the Digital Lifestyles Center, University of Essex, 
UK, and e-Learning Lab, Shanghai Jiaotong University, 
China, used skin sensors, along with blood volume pressure 
monitors and electroencephalograph, to chart the role of 
emotion in learning and how that feedback can be used to 
improve the learning process (Shen et al., 2009). Several 
other studies have explored the role of affect in learning by 
using skin conductance (Dragon et al., 2008; Arroyo et al., 
2009; Woolf et al., 2009; Hardy et al., 2013). 

STUDY PURPOSE AND RESEARCH 
QUESTIONS 

This pilot study addresses two research questions: (1) 
How are undergraduate students' engagement levels im¬ 
pacted as a result of pedagogical approach (dialogue, movie, 
and lecture) about climate change? and (2) How are 
undergraduate students' understanding, confidence, and 
perceptions of climate change altered as a result of the 
multipronged pedagogical approach? We hypothesize that 
undergraduate students' in-class engagement (as reflected in 
the skin conductance data or GSR) will increase as active 
learning is increased in class and that undergraduate 
students' conceptual knowledge, confidence, and percep¬ 
tions about climate change will increase as a result of the 
combined pedagogical approach. In this exploratory study, 
we set out to determine whether we could measure changes 
in students GSR when using different pedagogical ap¬ 
proaches and then determine whether we could capture 
which teaching method was most engaging. Furthermore, 
we wanted to monitor how the combined approaches 
enhanced student learning about climate change. 

METHODS 

Context/Background 

Twenty-four undergraduate students were enrolled in 
an introductory environmental geology course at a large 
southern U.S. university. The average age of the students 
was 20.9 years, and the group included 5 females and 19 
males, with 21 of the 24 students pursuing a geoscience 
major. The majority of students self-reported as being 
affiliated with the Republican Party and holding Protes¬ 
tant/Christian belief systems (Fig. 2). Of the 24 students, we 
collected GSR data on 17 students, and all students 
completed pre- and postassessments, participated in class¬ 
room activities, and completed journaling assignments. 

Course Description 

The course was an introductory class to environmental 
geology. The goals of the course included (1) learning the 
various Earth systems and the major environmental pro¬ 
cesses that occur in these systems, (2) understanding the 
basic science of these processes and the evidence of global 
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Student Political Affiliation 



■ Republican 

■ Democrat 

■ Independent 

■ Green Party 

■ None 
Other 


Student Religious Affiliation 



■ Protestant 

Other 

Christian 

Atheist 

Agnostic 


None 


Other 


33 % 

FIGURE 2: Student political and religious affiliations. 


environmental change, (3) examining the social debates 
surrounding the major environmental issues of today, and 
(4) reflecting, discussing, and communicating about these 
issues in order to engage in environmental conversations. 
The specific learning objectives included identification of the 
Earth systems and their processes, explanation of the 
scientific underpinnings of major environmental issues 


facing society today, explanation of the perspectives of 
various sides of a debate around environmental issues, and 
communication of the science to others. The course focused 
on environmental issues that were covered in the context of 
global change, and information was delivered as it specif¬ 
ically related to climate change. The course was taught using 
a mixed delivery of teaching in which there were opportu- 
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nities during lecture for students to discuss content with 
partners and groups and to watch content appropriate 
movies. In addition, entire class sessions were dedicated to 
dialogue and student presentations about the course 
content. 

Lectures were delivered over a 75-min period and 
consisted of PowerPoint slide delivery with usually one or 
two active learning segments for students to discuss with 
their neighbor a question the instructor posed, such as "With 
your neighbor, share and write down three facts you read 
about climate change that impacts the southeastern United 
States (SEUS), from today's reading," "Write down all of the 
renewable energies you can think of and then share with 
your neighbor," or "Mitigation or adaption or geoengineer¬ 
ing? What are examples of each, and which do you think is 
the best strategy?" Lecture slides were provided to students 
after each class period using a class Web portal. 

Two movies were viewed during the course. The first, 
Earth the Operator's Manual (ETOM), was viewed over one 
class period, and the second, NOVA: Earth in Balance was 
viewed over two class periods. For our research, we collected 
data from ETOM. ETOM is a National Science Foundation 
(NSF)-funded documentary, hosted by geologist Richard 
Alley and consisting of 12 viewing segments (but all 12 can 
be viewed without interruption). It begins with Earth's 
climate history, including carbon dioxide (CO 2 ) in the 
atmosphere and the ice core record, and extends to 
anthropogenic influences, why the military considers climate 
change a threat, and the many viable sustainable energy 
options. The globe is traveled from China to Brazil to New 
Zealand to Spain to Texas, highlighting climate solutions 
these locales have made. 

Devoted class dialogues (40-75 min) were allotted 
during three class periods to discuss current readings and 
broader perspectives on climate change topics, although 
other smaller-scale and more focused discussions occurred 
during some lecture periods. Students were asked to work in 
a group no larger than five people and provide a discussion 
sheet. Students self-assembled their groups and self- 
assigned roles. One person was to serve in the facilitator 
role to lead the discussion, and another was to serve as the 
group note-taker. Before conversations began, students were 
asked to follow the Golden Rules of Dialogue (Nash et al., 
2008). These rules included the following: 

• Listen to others as we want to be listened to 

• No matter how outrageous a point of view might first 
appear, we must always grant it the right to be heard 
and understood 

• Be willing to find the truth in what we oppose and the 
error in what we espouse.. .at least initially 

Students were provided guiding questions, such as "Do 
you think climate change is happening? Why/Why not?" 
"What evidence do you have for your stance?" "What are the 
potential impacts of Earth's changing climate?" "What are 
some of the possible solutions to addressing climate 
change?" "Who might be the most impacted by a changing 
climate?" "What is the role of the geoscientist in solving 
these environmental problems?" Conversations were re¬ 
corded via a tape recorder. At the end of each dialogue, 
student groups submitted their discussion notes to the 
instructor. 


Classroom journals were kept by each student. Each 
month (4 mo total) students submitted a minimum of four 
entries. Two of the entries described personal reflections that 
included (1) what they learned about course-related topics, 
(2) what they want to know about course-related topics, (3) 
and a description of their perspective or opinion about 
course-related topics, including any changes that have 
occurred over the course. The other two entries included 
descriptions of documented conversations the student had 
with others, outside of class, about course-related topics. 

Data Collection and Analysis 

Human subjects' research approval was obtained by the 
appropriate institutional review boards, and participant 
consent was obtained for this research project. Thematic 
analysis of journal responses and transcripts, descriptive 
statistics, and parametric tests were employed during 
analysis of the results. Specifics about each of the 
instruments and methods employed, including trustworthi¬ 
ness, validity, and reliability, are described in further detail 
below. 

Skin Conductance 

Skin conductance readings were measured on a subset 
of students, due to limited number of sensors, during each 
class activity. Students were randomly selected during the 
initial class to wear hand sensors. For the following classes, 
students were randomly selected again; however, those 
subjects who had worn it for a different treatment were 
prioritized in order to ensure that at least some of the 
students would wear the hand sensor during all three 
treatments. A total of 17 students were measured during the 
classes, of which 6 wore the device during all three activities 
(movie, lecture, and dialogue). Students were provided an 
Affectiva Q Palm Sensor at the beginning of class and asked 
to put it on immediately. Once the sensor made skin contact, 
it automatically began collecting data. Students were asked 
to place it on their nonwriting hand and not to press on the 
unit or make unnecessary movements while wearing the 
device. At the end of class, students returned the device to 
the instructor, who kept notes on the timing of the 
classroom activity events as the class progressed for later 
data comparison to student GSR. The data from the Q 
sensor was then downloaded to the Q software program 
provided by the sensor manufacturer. Data were collected 
every 0.125 s over the period of the classroom activities. The 
large dataset was then plotted, a second-order, 30-s 
Savitzky-Golay smoothing function was applied in MatLab 
(Version 2013b), and t-tests on aggregated data were 
performed in the Statistical Package for Social Sciences 
(Version 21.0). The hand sensors collected not only skin 
conductance but also temperature and accelerometer data. 
The external influences during class on student skin 
conductance outside of the teaching stimuli were recorded. 
Limited temperature (0.1°C) changes were measured during 
each class, and students were all sitting in their chairs during 
each activity, with accelerometer data showing minimal 
movements. If students physically moved to arrange 
themselves in student groups, these data were omitted from 
the analysis to ensure movement was not a factor. As such, 
temperature and movement were not shown to be external 
factors on the collected skin conductance data. However, 
other variables such as student mood and out of class 
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variables can potentially be factors causing individual, day- 
to-day variances. However, these variables cannot be 
controlled in such an experiment and thus should be treated 
as a limitation of this research. 

Pre- and Post-Climate Change Content Knowledge, 
Confidence, and Perceptions 

Pre- and post-climate content, confidence, and percep¬ 
tion gains were measured using a 17-question Climate 
Concept Inventory with 10 multiple-choice content items 
and 7 (1 confidence and 6 perception) Likert scale items. The 
instrument was developed through modification of existing 
and applicable items inventoried from the American 
Association for the Advancement of Science (http:// 
assessment.aaas.org/), the Geoscience Concept Inventory 
(Libarkin and Anderson, 2005), the Yale Project on Climate 
Change Communication (Leiserowitz, 2008; Leiserowitz and 
Smith, 2010), and the Climate Stewardship Survey (Walker 
and McNeal, 2013) through the ongoing NSF-funded 
EarthLabs project (http://serc.carleton.edu/earthlabs/index. 
html). Selected items were then modified and aligned to the 
broad and systems-oriented climate change learning goals of 
the EarthLabs climate modules (Ledley et al., 2012). Items 
were validated through expert climate and curriculum 
developer review, in addition to item analysis for each 
question. The reliability of the instrument during testing 
with 245 upper-level high school students showed an overall 
0.67 Cronbach alpha score, a moderate level of reliability and 
acceptable for most exploratory research (Ravid, 1994). 
Although the instrument was originally designed for 
upper-level high school students, we have applied it to 
introductory college students, as we assumed that this 
college population was not significantly different from the 
upper-level high school population. Content-related items 
loaded on a single factor during principal component 
analysis showed a discrimination of more than 0.2 and a 
difficulty from 0.1 to 0.9. Since only one factor was 
represented for which all items loaded on this factor, the 
items are interpreted as similar constructs and are thus 
reliable replicates of the assessed climate change content. 
Demographic data were collected at the end of the surveys to 
minimize stereotype threat. Students took the pretest at the 
beginning of the course, prior to the coverage of the content 
in class, and again at the end of the course, prior to the final 
examination. Statistical analysis of pretest and posttest 
differences were conducted using a paired Student's f-test, 
and assumptions of the test were satisfied except for the 
assumption of randomly selected population, because our 
student population was self-selected to enroll in an 
introductory geology course that consisted largely of geology 
majors. 

Journal Reflections and Dialogue Transcriptions 

Student journals and dialogue transcriptions were 
analyzed to identify themes that emerged from the 
responses and identify direct quotes that were representative 
of the larger group. Furthermore, transcripts established 
whether students were on task during discussions so that the 
hand-sensor data could be validated as representative of 
course content rather than unrelated discussion. Each 
student group was audio recorded during classroom 
dialogues, and transcripts were made by a third party. Four 
monthly journals per student were analyzed for content 
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TABLE I: Hand-sensor GSR means for each of the teaching 
approaches. 1 


Student 

Lecture Mean 
GSR (/is) 

Dialogue Mean 
GSR (/is) 

Movie Mean 
GSR (/is) 

1 

1.63 

3.23 

5.08 

2 

4.23 

7.13 

9.12 

3 

1.98 

5.44 

4.59 

4 

5.73 

5.69 

9.30 

5 

13.47 

12.89 

14.62 

6 

17.23 

13.19 

15.55 

7 

2.26 

2.96 

N.D. 

8 

4.84 

N.D. 

3.40 

9 

3.45 

N.D. 

7.44 

10 

14.19 

N.D. 

7.97 

11 

12.09 

N.D. 

N.D. 

12 

8.57 

N.D. 

N.D. 

13 

12.16 

N.D. 

N.D. 

14 

1.88 

N.D. 

N.D. 

15 

1.63 

N.D. 

N.D. 

16 

2.75 

N.D. 

N.D. 

17 

4.81 

N.D. 

N.D. 

Average 

6.66 

7.21 

8.56 


J N.D. = no data. 


themes at the passage level, where topics and representative 
phrases were extracted. Qualitative data can have different 
trustworthiness than quantitative research that has estab¬ 
lished validity and reliability measures. However, as the 
qualitative data were analyzed in this research, researchers 
took care to continuously compare individual responses to 
the larger whole to be sure specific examples and themes 
selected were representative. 

RESULTS 

Student Engagement 

Results from GSR indicated that students were more 
engaged during movie viewing (M = 8.56 /is) and dialogue 
(M = 7.21 /.is) than during lecture (M = 6.66 /is), where 
differences among each instructional approach was mea¬ 
sured (Table I). Statistical analyses with students engaged in 
more than one activity showed differences in the engage¬ 
ment level across the three treatments. 

GSR time plots indicated that the individual student 
responses show different patterns among the three teaching 
approaches (Fig. 3). Lectures illustrated several more 
engagement cycles, with students alternating between tonic 
and phasic components more frequently than dialogue and 
movie-viewing activities. Furthermore, student engagement 
during lecture appeared to correspond with classroom 
events, such as group work and topics (e.g., impacts on 
society; Fig. 4). Whereas approximately 0-30 min represent¬ 
ed traditional lecture covering the factors effecting climate 
change, 30-40 min represented a student in a class activity 
where students were to work with their neighbor to discuss 
the previous day's reading assignment, 40-52 min repre- 
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Student#! 



Student #2 



FIGURE 3: Three student example GSR responses for each of the three teaching approaches (lecture, dialogue, and 
movie). Some dialogue events were shorter (40-70 min) than other classroom activities due to the nature of the 
activities during different class periods. 


sented traditional lecture about the evidence of climate 
change, and 52-75 min represented lecture that covered the 
impacts of climate change, including local and regional 
impacts. Alternatively, during dialogues, student responses 
appeared to be more constant, with less tonic phases (Fig. 5), 


where continuous conversation occurred throughout the 
duration of a class of 0-40 min. During movie viewing, 
student engagement corresponded to content, similar to 
lectures; however, the slow upward ramp pointed to a 
building effect, with a gentle slope, peaking toward the end 


Student #2 Lecture GSR 



FIGURE 4: Student 2 GSR response for the classroom lecture activity. 
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Student #2 Dialog GSR 



FIGURE 5: Student 2 GSR response for the classroom dialogue activity. 


of the movie, which climaxed at the coverage of solutions 
(Fig. 6). Portions of the movie's content aligned with 
increases in student engagement, where peaks in phasic 
responses were evident, such as from approximately 11-13 


min (humans and energy), 18-22 min (fossil fuels), 25-28 
min (CO 2 and the atmosphere), 35-42 min (China), 42-58 
min (renewables and Texas), and 62-70 min (solutions), and 
valleys were indicated between these time periods. The first 


Student #2 Movie GSR 



FIGURE 6: Student 2 GSR response for the classroom movie activity. 
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TABLE II: Student pre-climate change (pre) and post-climate change (post) content scores (%), standard deviation (std dev.), gains 
(%), and effect size on multiple-choice items. 


No. 

Question 

Mean Pre (%) 

Std Dev. 

Mean Post (%) 

Std Dev. 

Gain (%) 

Effect Size 

Q1 

Where are glaciers found today? 

52.50 

26.20 

68.75 

21.87 

16.25 

0.32 

Q2 

What is a positive feedback loop in a 
system? 

76.47 

44.72 

93.75 

25.00 

17.28 

0.23 

Q3 

Which of the following will occur if the 
amount of sea ice decreases? 

70.59 

47.87 

87.5 

34.16 

16.91 

0.20 

Q4 

Which of the following factors can change 
the Earth's cryosphere? 

62.50 

38.7 

79.7 

33.19 

17.20 

0.23 

Q5 

Which of the following contributes to the 
transfer of thermal energy from place to 
place around the Earth? 

88.24 

34.16 

93.75 

25.00 

5.51 

0.09 

Q6 

How does the temperature of air on Earth 
increase? 

58.82 

51.23 

75.00 

44.72 

16.18 

0.17 

Q7 

Which of the following could be a possible 
cause of the Earth going through a warming 
period? 

57.80 

19.83 

88.81 

27.00 

31.01 

0.55 

Q8 

What are greenhouse gases? 

23.53 

40.31 

25.00 

44.72 

1.47 

0.02 

Q9 

Besides water, what do plants use to make 
the sugar that is used to build plant 
structures? 

58.82 

50.00 

80.00 

41.40 

21.18 

0.22 

Q10 

How has the amount of carbon dioxide in 
the atmosphere changed since the start of 
the Industrial Revolution 150 years ago? 

88.24 

34.16 

100.00 

0.00 

11.76 

0.23 


approximately 11 min of the class period, students were 
engaged not in the movie but rather in other, more general 
classroom activities, such as class attendance record keeping, 
announcements and reminders about the course, and movie 
overview. 

Student Climate Content Knowledge, Confidence, and 
Perceptions 

Results indicated that students' overall climate content 
knowledge gains increased 16% on average for the 10 
content items (Table II). "Where are glaciers found today?" 
had a 16% gain, and "Which of the following could be a 
possible cause of the Earth going through a warming 
period?" had a 31% gain. These items had an effect size of 
0.32 and 0.55, respectively. All other content items showed 
gains ranging from 1.47%-21.8% and effect sizes from 0.02 
to 0.23 (Table II). A Student's f-test for the overall content 
portion of the assessment, as well as the overall perceptions 
or confidence portion, showed that the pretest and posttest 
scores for the class (N = 24) were statistically significant (p < 
0.05). 

Results indicated that students' overall confidence and 
perceptions of climate change were increased by 10% on 
average for the seven Likert scale (1-4) questions. Four of 
the seven items showed effect sizes of 0.26-0.59, with gains 
of 0.28-0.81 (Table III). The four items included "How 
confident are you in your responses to the previous question 
set?" "Which comes closer to your view?" (which asked 
student about whether scientists agreed or disagreed with 
the state of Earth's climate changes), "Personally, how well 
informed do you feel you are about how the Earth's 'climate 
system' works?" and "How much have you thought about 


the Earth's climate system?" All other items showed effect 
sizes of 0.12-0.22, with gains of 0.25-0.30. Furthermore, 
when students were asked whether they were aware of any 
evidence of climate change and to state the evidence, in an 
open-ended response question, 56% of the students were 
able to state at least one example in the pretest; however, 
81% stated an example posttest, indicating increased 
awareness of the evidence of climate change. 

Student Dialogues and Journal Reflections 

Dialogue sessions and individual journals provided the 
opportunity to examine more deeply the attitudes and 
perceptions students held, as well as the self-reflected 
knowledge gains students made about climate change. 

During classroom dialogues, especially early in the 
course, it was clear groups did not have consensus about 
the causes of climate change. Some groups were more 
willing to consider anthropogenic contributions (example 1) 
than were others (example 2). Some groups discussed how 
those around them viewed climate change and how being 
located in the South seemed to influence people's opinions 
(example 3). All groups accepted that the Earth's climate was 
changing and could provide evidence of the changes 
(example 4). Some groups offered solutions that were tied 
to needed changes of individuals and society (example 5), 
whereas other groups did not see how any human action 
would alter the state of the climate (example 6). The groups 
that described anthropogenic causes also illustrated solu¬ 
tions that required individual and societal action (e.g., 
examples 1 and 5); however, groups that did not correlate 
climate change to anthropogenic causes did not consider any 
potential solutions to the problem (examples 2 and 6), at 
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TABLE III: Student pre-climate perceptions (pre) and post-climate perceptions (post) scores on Likert scale items (1 = strongly 
disagree, 4 = strongly agree), standard deviation (std dev.), gain, and effect size. 


No. 

Question 

Mean Pre 

Std Dev. 

Mean Post 

Std Dev. 

Gain 

Effect Size 

Q1 

How confident are you in your responses to the 
previous question set? 

2.88 

0.5 

3.25 

0.58 

0.37 

0.32 

Q2 

Which of the following statements about global 
warming over the past 50 years most closely 
reflects your view point? 

2.77 

0.86 

3.06 

0.25 

0.30 

0.22 

Q3 

Which comes closer to your view? [Scientist 
Climate Change Agreement] 

3.53 

0.63 

3.81 

0.40 

0.28 

0.26 

Q4 

Are you aware of any evidence of global climate 
change? 

2.47 

0.81 

2.75 

0.58 

0.28 

0.20 

Q5 

Personally, how well informed do you feel you 
are about how the Earth's "climate system" 
works? 

2.71 

0.48 

3.31 

0.48 

0.61 

0.53 

Q6 

How concerned are you about the possibility of 
global climate changes? 

2.38 

0.89 

2.6 

0.96 

0.25 

0.12 

Q7 

How much have you thought about the Earth's 
climate system? 

2.88 

0.62 

3.69 

0.48 

0.81 

0.59 


least at the early stages of the course. Example 4 is an excerpt Student 1: "I don't think that we're the sole cause, hut we 

from Student 2's (Figs. 4-6) group discussion and illustrates could possibly be adding to it." 

the content of the conversation with which the student was 

engaged. The level of attention in the examples provided Student 4: "I think we're definitely adding to it." 

were typical of the student conversations and showed that 

the hand-sensor data collected during dialogue sessions 

captured on-task conversations. Example 2 


Example 1 

Group Leader: ''Do you think climate change is happening? 
Why/why not?" 

Student 2: "The Earth itself goes through a cycle. That's part 
of the climate change." 

Student 1: "Right. There's periods of transgression and 
regression." 

Student 3: "I think that what’s going on right now is 
natural. But I do think that there's potential for what 
humans are doing too." 

Group Leader: ".Agree something's happening. We don't 
know if that's natural, manmade, or just ..." 

Student 3: "Maybe, it's both." 

Group Leader: "It could be both." 

Student 1: "My personal opinion is that it's an elitist scam 
to tax us more." 

Student 3: "If you look through the Earth's crust, it went 
through the same thing we're going through. Even though 
there are some manmade emissions in the air, it's still going 
through the same thing, the same cycles. That's just what I 
think about that. It's just going to go through that cycle." 


Group Leader: "Do you think climate change is happening? 
Why/why not" 

Student 3: "After all the classes I had here, I know that 
humans in some senses have changed the climate, but for the 
most part, it's really just been naturally increasing the heat. 
In some of my classes they showed us how in the past there's 
always been a rise and fall in the major temperatures and 
stuff, and it's just been like a natural process, aide. We 
might have played a small part in it, but it's nothing that's 
going to make the biggest difference. Honestly, I think it's 
just going to be all natural." 

Student 1: "My thing is how Venus has real big greenhouse 
gases? Nobody lives on Venus. That's naturally occurring. If 
that's naturally occurring over there, how are we making it 
worse ... ? We coidd be making it worse, but it's mostly like 
he says, mostly natural." 

Group Leader: "I took an Earth Science class and a 
meteorology class last semester, and the Earth's climate is 
changing every second of the day. It has been for thousands 
of years. That's pretty good evidence that it changes." 

Student 1: "When you look at history and you're like, 'Oh, 
there have been multiple ice ages,' obviously climate change 
is real." 

Group Leader: "Even the data out there does not make any 
clear, concise ... even Al Gore flies around in his nice jets. 
He's going and preaching about global warming, but is 
causing it all at the same time." 
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Student 2: "I think they blow it out of proportion a little bit, 
because blaming the people when, in reality, it is a natural 
process." 

Example 3 

Group Leader: "Do you think climate change is happening? 
Why/why not?” 

Lead Student: "Well, everybody thinks it's happening. It's 
just ... most people think it's happening." 

Student 1: "Well, most people who are involved in it 
acknowledge that it's happening, but there are still some 
people that are actually involved and outspoken that 
completely deny it at all, and then some people just think 
the entire idea of global warming is preposterous. They just 
don't believe there's anything happening at all." 

Student 2: "My grandparents think that climate change is 
made up and global warming doesn't exist. They think it's 
like a fairy tale thing." 

Student 1: "Especially around here in the South. Apparently 
[state where student is from] used to have a lot worse winters 
and you'd think older people would remember that .... 7 
guess it just really takes one bad winter to traumatize them." 

Group Leader: "Yeah, when temperatures get down into the 
twenties, you're like, [jokingly] Global warming? What? 
[laughs]" 

Example 4 

Group Leader: "Do you think climate change is happening? 
Why/why not?" 

Student 1: "Yes." 

Student 2: "Yes." 

Student 3: “Well that's easy." 

Student 4: "It's happening." 

Student 3: "Yeah, it's happening." 

Student 4: "There's loads of evidence. Like the sea level rise, 
the melting of the ice caps ... the carbon storage in the ice, 
and the severe thunderstorms." 

Student 3: “The evidence is everywhere." 

Student 4: "But like, there's just evidence everywhere." 

Example 5 

Group Leader: "What are some possible solutions to 
addressing climate change?" 


Student 2: "I think the issue woidd be whether or not to use 
adaptation or mitigation. I think that if everybody knew 
about it they would know what to think about it. They could 
have better opinions." 

Student 1: "I think that's the real problem, really. People just 
don't have opinions about stuff one way or the other, 
because they just get informed by one person, like the news or 
something." 

Student 2: “You can make them learn, bid you can't make 
them care." 

Student 1: "That's true." 

Student 3: "We could attempt to get people to stop polluting. 
It would take a lot of convincing." 

Student 4: "It could go back to the tax for some of the policies 
we're wanting to do. Like if they tax these people who are 
releasing more pollution than others, if they tax them more 
then it will make them want to stop." 


Example 6 

Group Leader: "What are some possible solutions to 
addressing climate change?" 

Student 1: "I thought we all agreed that climate change was 
pretty much natural. I don't understand. I don't know any 
solutions to the natural world." 

Student 2: "You can’t really." 

Student 4: "It's dependent on the extent of what you think 
climate change is, and the cause of it. There are people 
that think it is only because of humans, and that kind of 
thing, which I think we're all on agreement where it's 
probably a natural thing. Granted, we may add a little bit 
to it." 

The dialogues provided insights about student perspec¬ 
tives that would otherwise be difficult to extract. In 
conversation with their peers, students were able to state 
their own perspectives while hearing the perspectives of 
others, simultaneously incorporating the content (e.g., 
mitigation, adaptation, and causes and evidence of climate 
change) received from the course into their conversations 
throughout the semester. Furthermore, in journaling, they 
could reflect on their classroom experiences about topics 
covered in the classroom during dialogues, and the 
corresponding lectures and movies were often the impetus 
for the themes students discussed in their journals. The 
journal entries themes included the following: the water 
cycle and cryosphere, local issues such as the Deepwater 
Horizon oil spill and impact on the Gulf Coast, water 
resources management, population growth, conservation of 
natural resources, recycling and landfills, nuclear and green 
energy, drivers of global climate change, evidence of climate 
change, and impacts of climate change. During these 
monthly journal reflections and conversations with others 
outside of class, students commented about the amount of 
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knowledge they gained during the course. One student 
explained, "Just in a few weeks of classes I have learned ten¬ 
fold of the amount I knew before [about climate change]." 
Another student stated, "I had thought before about 
evidence of climate change, but I never really knew exactly 
how much is out there. Evidence of the heating of the Earth 
is all around us." A third student added, "I have done some 
research here and there and heard conversations about 
climate change, and global warming, but unfortunately, if I 
am being honest, I could never have educated anyone well 
on the subject. It is a big deal, but I was very unaware of the 
details." These comments highlight that the mixed approach 
of lecture, dialogue, and movies have positively impacted 
student's self-reflections about their own knowledge about 
climate change. 

Many students recognized that knowledge was not 
enough and that their newly formed knowledge needed to 
translate into practice: "It is important to know how I can 
protect our good Earth. Now that I am moving away from 
the ignorance is bliss stage, I want to learn quickly and help 
prevent quickly," "A paradigm shift needs to occur to bring 
about a new standard of living that has less to do with 
material wealth and more to do with environmental 
sustainability goals for ourselves and our society," and "In 
my opinion, small changes in our everyday lives can make a 
big difference in changes to our climate." These responses 
illustrated that students were applying their knowledge to 
everyday life and saw it as relevant and valuable for their 
own future actions. 

Other students recognized the value in knowing about 
the issue for their future career: "I believe that these 
problems are really important, so I find it very important 
to know all of the bases I need to cover in my future as a 
geoscientist" and "When I first began this major, my goal 
was simply to get into an oil company and make the big 
bucks. This goal has not changed completely but.. .1 can't 
help but morally disagree with them." This last statement 
specifically addresses how the perceived value systems 
between the energy sector and environmental sustainability 
practices were perhaps an area of emotional conflict for this 
student. Other students noticed how dialogue activities gave 
them insights about how important it is to be an informed 
citizen. One stated, "Sometimes witnessing a classmate's 
response to the different issues reminds me to learn about 
the issues before I try to make an opinion." Another 
recognized the need to communicate with others: "With the 
knowledge I have gained in this class, I will be able to 
discuss environmental issues with my friends and family." 
These responses showed that the students were applying 
their knowledge to careers and communication activities that 
they saw as important and valuable. 

The dialogue transcript and journal responses illustrated 
that self-reflected knowledge gains supported the quantita¬ 
tive pre- and post-climate content assessment and pre- and 
post-climate perceptions findings. Transcript results illus¬ 
trated that students were on task during these activities, 
validating collected GSR data. Overall, the combined 
approach of lecture, dialogue, movie viewing, journal 
reflection, and conversation with others in and outside of 
class provided students the opportunity to learn about 
climate change through a variety of strategies that promoted 
engagement. 
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DISCUSSION AND IMPLICATIONS 

This study presents GSR as a method to measure 
undergraduate student engagement in classrooms empha¬ 
sizing climate science. Results indicated that movie viewing 
was the most engaging pedagogical technique when 
learning about climate change. However, it is difficult to 
know exactly why movies were more engaging then lectures, 
given that both activities have the student in a more passive 
role and a receiver of information. It is possible that since 
movies are designed to be stories, they are more engaging 
than the typical lecturer. Music in the movies may also play a 
role on engagement, as well as the visual nature of most 
movies, which may promote more emotional responses than 
the other two approaches in this study. It is beyond the 
scope of this pilot project to determine why these differences 
occurred, but it is certainly a recommended direction for 
future research. Student content knowledge, confidence, and 
perception gains about climate point to the value of the 
combined pedagogical approach of utilizing movies, lectures, 
and classroom dialogues when teaching about climate 
change. Furthermore, student dialogues illustrated how 
students were trying to grapple with the idea of climate 
change, what evidence was known and accepted, and 
whether anthropogenic or natural causes (and solutions) 
were the dominant mode of operation for the student 
groups, where differences of opinion among groups were 
observed. Student reflections of their own learning showed 
enhanced self-perceived awareness and knowledge about 
climate change where students were given the opportunity 
to expand further on topics covered in class, align their own 
ideas with the content in the course, and discuss these ideas 
with others outside of class. The multiple pathways of 
acquiring knowledge (movies and lectures), sharing among 
peers (dialogue), and self-reflection (journaling) provided a 
unique learning environment to the students. Both content 
knowledge and perceptions showed marked improvement, 
and the journal reflections indicated a deep sense of 
commitment toward climate change issues. Behavioral 
change is hard to measure and beyond the scope of this 
study, but climate change knowledge and awareness can be 
argued to be prerequisites for individual behavior changes 
(O'Connor et al., 1999; Bord et al., 2000; Semenza et al., 
2008). 

One of the limitations of this study was the small sample 
size. Also, the study was conducted in a classroom setting 
(less control than a laboratory), where it was difficult to align 
the exact timing of events with the GSR measures. 
Therefore, the GSR results presented are coarse. Further¬ 
more, since the course had a high number of students who 
were geoscience majors, this may prebias the population, 
which is already scientifically inclined. A different response 
might occur in a nonmajors course. In addition, student 
emotional state may change day to day and is difficult to 
control for in this study, where repeated measures of student 
engagement occurred over time. However, despite these 
limitations, this study is the first of its kind to utilize GSR in 
an undergraduate classroom when teaching and learning 
about climate change. This study shows that GSR holds 
great potential in measuring student engagement in the 
classroom. However, more work is needed to understand 
the nuances of the approach. 

Teaching and learning about climate change should 
utilize a variety of approaches in order to engage students 
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about the often polarizing issue of climate change. In an era 
when educators are called to train future scientists to 
communicate effectively with the public while simultaneous¬ 
ly depolarizing the issue, we must recognize comprehensive 
teaching strategies in order to make significant inroads in 
building a climate-literate society that is able to grapple with 
the complexity of the phenomenon and develop practical 
solutions. Science, technology, engineering, and mathemat¬ 
ics education, in general, should use multiple approaches 
that focus on active (Prince, 2004) and group (Cohen, 1994) 
learning, and climate change education is no different. This 
research shows through GSR, for the first time, that there are 
measurable engagement differences among the differing 
pedagogical techniques (e.g., movies, lecture, and dialogues) 
applied in the undergraduate classroom. When these 
techniques are employed together, significant learning gains 
can result. 
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